Unstructured Image Mosaics

Unstructured Image Mosaics Rahul Garg Chair of the Supervisory Committee: Professor Steven M. Seitz Computer Science & Engineering Image mosaics stitch photos into a single composite with a wide field of view. They are easy to create and can be panned by dragging the mouse, thus enabling simple and effective photorealistic visualizations. However, they are restricted to structured input where the camera motion is limited and the appearance variation across photos can be controlled. In my thesis, I extend mosaics to unstructured cases that include more general camera motion, appearance variation and Internet photo collections, enabling visualization of more complex scenes. I first develop a mosaicing technique for a general class of photo collections subsuming rotational mosaics. In contrast to prior approaches that stitch a single static mosaic, my approach dynamically composites a mosaic based on the current viewpoint as the user navigates, allowing for distortion-free mosaicing and a broader range of camera motion. Exposure differences, parallax, misalignment between photos and scene motion lead to stitching artifacts in static mosaics. My dynamic approach avoids these artifacts and enhances realism by preserving scene motion and changes in brightness. In the latter part of my thesis, I focus on highly unstructured collections comprising tourist photos downloaded from the Internet that are not captured with the intention of creating a mosaic. First, I describe an algorithm to discover rotational panoramas (photos taken from nearly the same viewpoint) and orbits (photos looking at a common object), within these collections. These photo sets can then be browsed by dragging the mouse like traditional mosaics. Second, I focus on extreme variation in appearance of these photos. I prove that any photo of a scene can be represented as a linear combination of a set of basis photos. I show theoretically and empirically that under suitable assumptions, for a scene with kn distinct orientations and kρ different materials, knkρ basis photos are sufficient to span the space of all possible photos of the scene. I then describe a method to robustly compute these basis photos from Internet photos and show novel applications like removing people and expanding the field of view of a photo.